Optimization with Extremal Dynamics 
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We explore a new general-purpose heuristic for finding high-quality solutions to hard opti- 
mization problems. The method, called extremal optimization, is inspired by self-organized 
criticality, a concept introduced to describe emergent complexity in physical systems. Extremal 
optimization successively replaces extremely undesirable variables of a single sub-optimal so- 
lution with new, random ones. Large fluctuations ensue, that efficiently explore many local 
optima. With only one adjustable parameter, the heuristic's performance has proven com- 
petitive with more elaborate methods, especially near phase transitions which are believed to 
coincide with the hardest instances. We use extremal optimization to elucidate the phase transi- 
tion in the 3-coloring problem, and we provide independent confirmation of previously reported 
extrapolations for the ground-state energy of ±J spin glasses in d = 3 and 4. 
PACS number(s): 02.60.Pn, 05.65.+b, 75.10.Nr, 64.60. Cn. 
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Many natural systems have, without any central- 
ized organizing facility, developed into complex struc- 
tures that optimize their use of resources in sophisti- 
cated ways Q. Biological evolution has formed efficient 
and strongly interdependent networks in which resources 
rarely go to waste. Even the morphology of inanimate 
landscapes exhibits patterns that seem to serve a pur- 
pose, such as the efficient drainage of water [||, f|. 

Natural systems that exhibit self-organizing qualities 
often possess a common feature: a large number of 
strongly coupled entities with similar properties. Hence, 
at some coarse level they permit a statistical description. 
An external resource (sunlight in the case of evolution) 
drives the system which then takes its direction purely 
by chance. Like descending water breaking through the 
weakest of all barriers in its wake, biological species are 
coupled in a global comparative process that persistently 
washes away the least fit. In this process, unlikely but 
highly adapted structures surface inadvertently. Optimal 
adaptation thus emerges naturally, from the dynamics, 
simply through a selection against the extremely "bad" . 
In fact, this process prevents the inflexibility inevitable 
in a controlled breeding of the "good" . 

Various models relying on extremal processes have 
been proposed to explain the phenomenon of self- 
organization Q|. In particular, the Bak-Sneppen model 
of biological evolution is based on this principle |], ^) . As- 
suming an unspecified interdependency between species, 
it produces salient nontrivial features of pale-ontological 
data such as broadly distributed lifetimes of species, large 
extinction events, and punctuated equilibrium. 

In the Bak-Sneppen model, species are located on the 
sites of a lattice, and have an associated "fitness" value 
between and 1. At each time step, the one species 
with the smallest value (poorest degree of adaptation) 
is selected for a random update, having its fitness re- 
placed by a new value drawn randomly from a flat dis- 
tribution on the interval [0, 1]. But the change in fitness 



of one species impacts the fitness of interrelated species. 
Therefore, all of the species at neighboring lattice sites 
have their fitness replaced with new random numbers 
as well. After a sufficient number of steps, the system 
reaches a highly correlated state known as self-organized 
criticality (SOC) 0. In that state, almost all species 
have reached a fitness above a certain threshold. These 
species, however, possess punctuated equilibrium: only 
one's weakened neighbor can undermine one's own fit- 
ness. This coevolutionary activity gives rise to chain re- 
actions called "avalanches" , large fluctuations that rear- 
range major parts of the system, potentially making any 
configuration accessible. 

Although cocvolution does not have optimization as 
its exclusive goal, it serves as a powerful paradigm. We 
have used it as motivation for a new approach to op- 
timization ||. The heuristic we have introduced, called 
extremal optimization (EO), follows the spirit of the Bak- 
Sneppen model, merely updating those variables having 
the "worst" values in a solution and replacing them by 
random values without ever explicitly improving them. 

Previously, several heuristics inspired by natural pro- 
cesses have been proposed, notably simulated anneal- 
ing Q and genetic algorithms ]l0| . These have aroused 
considerable interest among physicists, in the develop- 



ment of physically motivated heuristics lllL 
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1, and 

in their applications to physical problems 14, |15|, fl6| . 
EO adds a distinctly new approach to the approximation 
of hard optimization problems by utilizing large fluctua- 
tions inherent to systems driven far from equilibrium. In 
this Letter we demonstrate EO's generality, the simplic- 
ity of its implementation, and its results, for a physical 
problem as well as for a classic combinatorial optimiza- 
tion problem. A tutorial introduction is given in Ref. p). 

The physical problem we consider is a spin glass |1S|. It 
consists of a d-dimcnsional hyper-cubic lattice of length 
L, with a spin variable x\ G { — 1,1} at each site i, 
1 < i < n (= L d ). A spin is connected to each of its 



2 



nearest neighbors j via a bond variable Jij G { — 1,1}, 
assigned at random. The configuration space f2 consist of 
all configurations S — (x±, . . . , x n ) 6 £1 where |f2| = 2™. 
We wish to minimize the cost function, or Hamiltonian 



(1) 



Due to frustration |l8| , ground state configurations S m - ln 
are hard to find, and it has been shown that for d > 2 the 
problem is among the hardest optimization problems [ fl9| . 

To find low-energy configurations, EO assigns a fitness 
to each spin variable Xi 



so that 
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Unlike in genetic algorithms, the fitness does not charac- 
terize an entire configuration, but simply a single variable 
of the configuration. In similarity to the Bak-Sneppen 
model, EO then proceeds to search fl by sequentially 
changing variables with "bad" fitness at each update 
step. The simplest "neighborhood" N(S) for an update 
consists of all configurations <S" e N(S) that could be 
reached from S through the flip of a single spin. After 
each update, the fitnesses of the changed variable and of 
all its neighbors are reevaluated according to Eq. (||) . 
The basic EO algorithm proceeds as follows: 



FIG. 1: Plot of the average energies obtained by EO for 
the ±J spin glass in d = 3 as a function of r. For each 
size n, 10 instances were chosen. For each instance, 10 
EO runs were performed starting from different initial 
conditions at each r. The results were averaged over 
runs and over instances. The best results are obtained 
at r moving slowly toward r — > 1 + for n — > oo. 



single parameter into the algorithm. We rank all the vari- 
ables Xi according to fitness Xi, i.e., find a permutation 
II of the variable labels i with 
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i.j [step fl2q)] is of rank 1, j = 11(1), 
and the best variable is of rank n. Now, consider a prob- 
ability distribution over the ranks k, 



P k ozk~ 



l<k<n, 
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1. Initialize configuration S at will; set Sbcst '—S. 

2. For the "current" configuration S, 

(a) evaluate Xi for each variable Xi, 

(b) find j satisfying Xj < Xi for all i, i.e., Xj 
has the "worst fitness" , 

(c) choose S'gN(S) so that Xj must change, 

(d) accept S :— S' unconditionally, 

(e) if C(S) < C(S hcst ) then set S bcst := S. 

3. Repeat at step (||) as long as desired. 

4. Return Sbest and C(S hest ). 



Initial tests have shown that this basic algorithm is 
quite competitive for optimization problems where EO 
can choose randomly among many S' G N(S) satisfying 
step (pc|), such as graph partitioning S. But, in cases 
such as the single spin-flip neighborhood above, focusing 
on only the worst fitness [step (pb|)] leads to a determinis- 
tic process, leaving no choice in step (2d). To avoid such 
"dead ends" and to improve results 



for a given value of the parameter r. At each update, 
select a rank k according to Pk- Then, modify step ( pc] ) 
so that the variable Xi with i — n(fc) changes its state. 

For r = 0, this "r-EO" algorithm is simply a random 
walk through f2. Conversely, for r — > oo, the process ap- 
proaches a deterministic local search, only updating the 
lowest-ranked variable, and is bound to reach a dead end 
(see Fig. |]). However, for finite values of r the choice of 
a scale-free distribution for Pf. in Eq. (||) ensures that no 
rank gets excluded from further evolution, while main- 
taining a clear bias against variables with bad fitness. 

For the spin glass, we obtained our best solutions for 
t — > 1 + , as shown in Fig. [j]. Generally, over many op- 
timization problems, the preferred value seems to scale 
slowly asr — 1 ~ 1/lnn for increasing n. Experiments 
with random graphs support this expectation pQ ], al- 
though the dependence on n can be practically negligible. 
For instance, at fixed r = 1.4, r-EO already reproduced 
many testbed results for the partitioning of graphs of 
sizes n = 1000 to n > 10 5 ||. 

We have run the r-EO algorithm with r = 1.15 on a 
large number of realizations of the Jij, for n = L d with 
we introduce a L = 5, 6, 7, 8, 9, 10, 12 in d = 3, and with L — 3,4, 5, 6, 7 



in d = 4. To reduce variances, we fixed | Y] J t - 7 - 1 < 1. 
For each instance, we have run EO with 5 restarts from 
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random initial conditions, retaining only the lowest en- 
ergy state obtained, and then averaging over instances. 
Inspection of the convergence results for the genetic al- 
gorithms in Refs. |l4|, Q suggest a runtime scaling at 
least as n 3 -n 4 for consistent performance. Indeed, using 
~ n 4 /100 updates enables EO to reproduce its lowest 
energy states on about 80% to 95% of the restarts, for 
each n. Our results are listed in Table 0. A fit of our 
data with ed(n) — ed{oo) + A/n for n —> oo predicts 
e 3 (oo) = 1.7865(3) for d = 3 and e 4 (oo) = 2.093(1) for 
d = 4. Both values are consistent with the findings of 
Refs. " 
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L5| [L6| , providing independent confirmation of 
those results with far less parameter tuning. 

In the future, we intend to use EO to explore the prop- 
erties of states near the ground state. As the results 
on 3-coloring in Ref. fj8| suggest, EO's continued fluc- 
tuations through near-optimal configurations even after 
first reaching near-optimal states may provide an efficient 
means of exploring a configuration space widely. 

To demonstrate the versatility of EO (see also 
Ref. 21 ), we now turn to a popular combinatorial op- 
timization problem, graph coloring p2fl . Specifically, we 
consider random graph 3-coloring p3[ , p4[ . A random 
graph is generated by connecting any pair of its n vertices 
by an edge, with probability p p5| . In A'-coloring, given 
K different colors used to label the vertices of a graph, 
we need to find a coloring that minimizes the number of 
"monochromatic" edges connecting vertices of identical 
color. We implement EO by defining for each vertex 
to be —1/2 times the number of monochromatic edges 
attached to it. Then, Eq. (||) represents exactly the cost 
function, counting the number of monochromatic edges 
present. As a simple neighborhood definition, at each 
update we merely change the color of one "bad" vertex 
selected according to t-EO [step (|c])]. 

As a special challenge, we have used r-EO for 3- 
coloring to investigate the phase transition that oc- 
curs under variation of the average vertex degree a — 
pn (2^, ^4|. Random graphs with small a can almost 
always be colored at zero cost, while graphs with large 
a are typically very homogeneous with a high but eas- 
ily approximated cost. Located at some point a cr it be- 
tween these extremes, there is a sharp phase transition 
to non-zero cost solutions. Such a critical point appears 
in many combinatorial optimization problems, and has 
been conjectured to harbor those instances that are the 
hardest to solve computationally |26) . Previously we have 
shown |27| that t-EO significantly outperforms simulated 
annealing near the phase transition of the bipartitioning 
problem of random graphs. 

Using EO we can estimate the value of a C rit for 3- 
coloring. To this end, we have averaged the cost EO 
obtains as a function of the vertex degree a. We gen- 
erated 10000, 5000, 1300, and 650 instances for n = 32, 
64, 128, and 256, respectively, for values of 3.6 < a < 6. 
Since n is relatively small and the runs were chosen to 
be very long (100n 2 updates), we found optimal perfor- 
mance at t — 2.7. Such excessively long runs were used 
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FIG. 2: Plot of the average cost as a function of the av- 
erage vertex degree a for random graph 3-coloring. Ac- 
cording to Eq. (|6J), the data collapse in the insert predicts 
«crit ~ 4.72, indicated by a vertical line. 

as part of a study to find all minimal-cost solutions for 
each instance, in order to determine their overlap (or 
"backbone" ) . Elsewhere [^8) we show that this backbone 
appears to undergo a first-order phase transition as con- 
jectured in Ref. p9[ . 

Finite size scaling with the ansatz 



(C)(a,n) -/ \{a-a ait )n l ' v 



(6) 



applied to the results depicted in Fig. g predicts a cr ;t ~ 
4.72(1) and v 1.53(5). These are the most precise esti- 
mates to date. The numerical value of v suggests that in 
fact v — 3/2, just as for the percolation transition of ran- 
dom graphs J25| and the transition in 3-satisfiability [^o| . 

In conclusion, we have presented a new optimization 
method, called extremal optimization due to its deriva- 
tion from extremally driven statistical systems. At each 
update step, the algorithm assigns fitnesses to variables 
Xi, and then generates moves by randomly updating an 
"unfit" variable. EO gives no consideration to the move's 
outcome. Large fluctuations in the cost function can ac- 
cumulate over many updates; only the bias against poor 
fitnesses guides EO back towards improved solutions. 

A drawback to EO is that a general definition of fitness 
for individual variables may prove ambiguous or even 
impossible. Also, each variable Xi could have a large 
number of states to choose from, as in fC-coloring with 
large K; random updates would then be more likely to 
remove than to create well-adapted variables (this is no- 
tably the case for the traveling salesman problem Q). 
And in highly connected systems (e. g. for a ^> 1 in 
iiT-coloring) , EO may be slowed down considerably by 
continual fitness recalculations [step (pa])]. 

However, extremal optimization is readily applicable 
to problems whose cost can be decomposed into contri- 
butions from individual degrees of freedom. It is easily 
implemented and, using very few parameters, it can prove 
highly competitive. We have shown this on the spin glass 
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TABLE I: EO approximations to the average ground-state energy per spin ed(n) of the ± J spin glass in d = 3 on the 
left, compared with genetic algorithm results from Refs. Jl4|, |lq| , and in d = 4 on the right (see also Ref. Jl6|). For 
each size n — L d we have studied a large number I of instances. Also shown is the average time t (in seconds) needed 
for EO to find the presumed ground state, on a 450MHz Pentium. 



L 


I e 3 (n) 


t 


Ref. ]14 


Ref. [ 


15i 




t 


3 


40100 


-1.6712(6) 


0.0006 


-1.67171(9) 


-1.6731(19) 


10000 


-2.0214(6) 


0.0164 
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40100 


-1.7377(3) 


0.0071 


-1.73749(8) 
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-1.7822(5) 
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777 


-1.7832(5) 


424. 


-1.78339(27) 


-1.7840(4) 
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30 


-1.7857(16) 


9720. 


-1.78407(121) 


-1.7851(4) 









Hamiltonian, obtaining d = 3 and 4 ground-state ener- 
gies that are consistent with the best known results. We 
have also used EO to explore the phase transition in ran- 
dom graph 3-coloring. Its results enable us to provide, 
by way of finite size scaling, the first sound estimates of 
critical values for this problem. 
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